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Abstract. Syntactic complexity is considered to be an important device for assessing 
the quality of writing in a second language (L2), as it indicates the diversity and 
complexity of production units or grammatical structures. This paper studies the 
development of Chinese college students’ syntactic complexity in essay writing 
by using an Automatic Writing Evaluation (AWE) tool, the Pigai system (www. 
pigai.org, which has been most widely used in China in the last ten years). The 
data analysis showed that the students’ syntactic competences in their final drafts 
outperformed that in their first drafts in three aspects: length of production unit, 


amount of subordination, and amount of coordination. 
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1. Introduction 


Although effective second language (L2) writing instruction requires the 
provision of regular feedback on students’ drafts (Bitchener & Ferris, 2012), for 
English as a Foreign Language (EFL) teachers, writing has long been seen as a 
tedious and unrewarding task (Hyland, 1990). With development of technology, 
AWE tools make it possible to solve this dilemma by providing instant holistic 
and diagnostic feedback. 
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Complexity, as one of the three important constructs of language development, along 
with accuracy and fluency (Larsen-Freeman, 1978), has been operationalized at 
lexical and syntactic levels (Wolfe-Quintero, Inagaki, & Kim, 1998). As mentioned 
in Lu and Ai (2015), “syntactic complexity has been commonly [considered] as 
the range of syntactic structures that are produced and the degree of sophistication 
of those structures (Ortega, 2003)” (p. 2). According to Lu (2010), four measures 
are commonly adopted for accessing syntactic complexity, namely, mean length 
of production units, amount of coordination, amount of subordination, and degree 
of phrasal sophistication. Based on Hunt (1965), the measure T-unit refers to “one 
main clause plus any subordinate clause or non-clausal structure that is attached to 
or embedded in it” (p. 141). 


A set of studies were conducted to examine the relationship between syntactic 
complexity and writing quality. Significant relationships have been identified 
between subordination and writing quality (Flahive & Snow, 1980), between finite 
clausal subordination and holistic grades (Homburg, 1984), and between length 
measures and composition quality (Homburg, 1984). Several longitudinal studies 
focus on learners’ development in syntactic complexity. For example, Casanave 
(1994) found after three semesters of instruction that Japanese English leaners’s 
writing had become more grammatically complex as measured by per clauses 
per T-units. The current study employed a full set of 14 measures provided in the 
L2 Syntactic Complexity Analyzer (L2SCA; Lu, 2010), an online computational 
tool used to analyze the syntactic complexity of the writing samples. The detailed 
information about the 14 different measures can be retrieved from Lu and Ai 
(2015, p. 3). 


2. Method 


The research question of the current study is whether AWE-based formative 
feedback has any positive effects on EFL learners’ syntactic complexity. 


This study was conducted at a college located in northern China, where English was 
taught as a one-week interval compulsory course. The study was carried out from 
October 2019 to November 2019 which lasted for five weeks with participation of 
66 non-English major freshmen. 


In the first week, students were assigned to write an argumentative essay entitled 
My View on the Role of Technology in Education on an AWE platform, the Pigai 
system. Students were required to complete at least 150 words of the writing task 
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within 50 minutes before revising their drafts based on the system’s corrective 
feedback and the instructor’s guidance. In the following four weeks, students were 
asked to revise their drafts for at least three times after being instructed on how 
to write an argumentative essay and how to develop topic-relevant arguments. 
The students did all the writing and revision through the system, and by which all 
the drafts were recorded and collected. Then the exported drafts were examined 
through the online computational tool, L2SCA, and statistical descriptions of 
syntactic complexity were collected and processed by using SPSS 24.0 software. 


3. Results and discussion 


Table 1 was produced to summarize the comparison of mean and standard 
deviation (SD) of the first and the final drafts in nine syntactic structures. It can be 
found that students’ performances in all nine syntactic structures in the final drafts 
outperformed than that in their first drafts. 


For other syntactic complexity indices, it is shown in Figure | that students 
produced more grammatically complex sentences based on the AWE system’s 


corrective feedback and the instructor’s guidance. 


Table 1. Descriptive statistics of first and final drafts 


Measures drafts Mean SD 
W First draft 190.61 58.28 
Final draft 317.86 DLL 
S First draft 11.44 4.26 
Final draft 17.05 5.00 
VP First draft 25.98 10.10 
Final draft 41.80 9.96 
Cc First draft 19.97 7.65 
Final draft 31.41 7.94 
T First draft 12.91 5.03 
Final draft 19.7 5.44 
DC First draft 5.88 3.71 
Final draft 10.11 4.80 
CT First draft 4.79 2.74 
Final draft 7.92 3.41 
CP First draft 5.27 3.58 
Final draft 8.06 4.00 
CN First draft 24.71 8.66 
Final draft 39.59 8.47 
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Figure 1. Comparison of students’ mean scores in first and final drafts 
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The results can be further explained in four ways. Firstly, students wrote much 
longer sentences, T-units, and clauses compared with first drafts, which shows 
development in writing quality according to findings of Wolfe-Quintero et al. 
(1998). Secondly, students’ progress in CP/T and CP/C indicates that they applied 
more coordinate phrases in T-units and clauses. Thirdly, their improvement in 
CN/C and CN/T demonstrates they can utilize more complex nominals in T-units 
and clauses. Lastly, that students did not make much progress in DC/C and DC/T 
reflects the fact that they did not employ dependent clauses in T-units or clauses. 
These findings confirm the results provided by Lu (2010). From the Table | and 
Figure | above, it is obvious that AWE-based argumentative writing has a positive 
effect on EFL learners’ syntactic complexity. 


4. Conclusions 


This study focuses on the students’ development in syntactic complexity based on 
the AWE system’s corrective feedback and the instructor’s guidance. The results 
show that AWE-based argumentative writing plays a positive role in improving 
EFL learners’ syntactic complexity. Online statistical analysis shows that the 
students’ syntactic competence in their final drafts outperformed that in their 
first drafts in the three following aspects: length of the production unit, amount 
of subordination, and amount of coordination. In the process of learner-computer 
interaction, students tend to modify their drafts with longer and more complex 
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sentences. We hope the study may provide other EFL instructors and learners 
with pedagogical implications for personalized learning in some similar computer 
assisted language learning teaching contexts. 
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